Back to the Future: Consistency-based Trajectory Tracking 


James Kurien 

Computational Sciences Division 
NASA Ames Research Center, MS 269-3 
Moffett Field, CA 94035 
jkurien@arc. nasa.gov 


P. Pandurang Nayak 

RIACS 

NASA Ames Research Center, MS 269-2 
Moffett Field, CA 94035 
nayak@cs.stanford.edu 


Abstract 

Given a model of a physical process and a sequence 
of commands and observations received over time, the 
task of an autonomous controller is to determine the 
likely states of the process and the actions required to 
move the process to a desired configuration. We intro- 
duce a representation and algorithms for incrementally 
generating approximate belief states for a restricted 
but relevant class of partially observable Markov de- 
cision processes with very large state spaces. The al- 
gorithm presented incrementally generates, rather than 
revises, an approximate belief state at any point by ab- 
stracting and summarizing segments of the likely tra- 
jectories of the process. This enables applications to 
efficiently maintain a partial belief state when it re- 
mains consistent with observations and revisit past as- 
sumptions about the process’ evolution when the belief 
state is ruled out. The system presented has been im- 
plemented and results on examples from the domain of 
spacecraft control axe presented. 


Introduction 

Given a model of a physical system and a sequence of 
commands and observations received over time, the task 
of an autonomous controller is to determine the likely 
states of the system and the actions required to move 
the system to a desired configuration. Focusing on the 
state identification question, a belief state is a proba- 
bility distribution over the possible states of a system. 
If the system has the Markov property, then the influ- 
ence pf a new command and observation upon the belief 
state can be integrated via Bayes’ rule. The updated 
belief state is a sufficient statistic, capturing within a 
single distribution all knowledge about the current state 
of the system contained within a history of commands 
and observations. The controller then makes use of the 
updated belief state in selecting an action. 

Example 1 Consider the spacecraft propulsion sub- 
system of Figure 1. The helium tank pressurizes the 
two propellant tanks. When a propellant path to either 
engine is open, the pressurized tanks force fuel and oxi- 
dizer into the engine, producing thrust. Not shown are 
valve drivers that control the the latch valves and a set 
of flow, pressure and acceleration sensors that provide 



Figure 1: Propulsion system schematic. 


partial observability. A model of a system specifies the 
modes of each component (e.g. a valve may be open, 
closed, stuck closed, and so on), behavior in each mode 
(e.g. a closed valve prevents flow), mode transitions 
(valves usually open when commanded, but stick closed 
with probability p) and connections between compo- 
nents (fuel flow into an engine is equal to the flow out 
of the attached fuel valve). 

Consider the problem of determining the likelihood 
of the possible states of this subsystem. Unfortunately, 
computing a belief state in general requires enumera- 
tion of the state space. The propulsion subsystem has 
38 components with an average of 3 states each. More 
complete spacecraft models capture 150 or more com- 
ponents averaging 4 states, yielding a state space of 
2 300 or more and making complete enumeration is im- 
plausible. One alternative is to track an approxima- 
tion whose computation does not require enumeration 
of the state space, ideally enumerating only the most 
likely portion of the belief state at each point in time. 
Livingstone (Williams ic Nayak 1996) tracks n approx- 
imately most likely states of the system by transition- 
ing a small number of tracked states by the transitions 
that are most likely, given only the current observa- 
tions. This approximation is extremely efficient and 
well suited to the problem of tracking the internal state 
of a machine, where the likelihood of the nominal or 
expected transition dominates and immediate observa- 
tions often rule out the nominal trajectory when a fail- 
ure occurs. The task then becomes one of diagnosing 
the most likely system transition, chosen from combi- 
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Figure 2: Evolution of a Valve Driver Unit and Valves 

nations of component transitions, that would be con- 
sistent with the unexpected observations. Using this 
technique, Livingstone is able to perform approximate 
state identification and reconfiguration of systems with 
hundreds of state variables. It has been applied to the 
control of a number of systems within NASA and is an 
integral part of the Remote Agent architecture demon- 
strated in-flight on the Deep Space I spacecraft in 1999 
(Bernard et ai 1998). Unfortunately, the true trajec- 
tory may not be among the most likely given only the 
current observations. Consider the following example. 

Example 2 Figure 2 illustrates a small system and two 
possible trajectories. The pump pressurizes the system 
and the valves, if open, allow a fluid flow. The valve 
driver unit commands the two valves in parallel via the 
data bus represented by dashed lines. The graph to the 
right represents the probability of two possible trajec- 
tories. The filled circles represent the true state of the 
system. At time 0 the VDU is off, the valves are closed 
and the pump is off. At time 0 the VDU is commanded 
on. For the sake of illustration, consider an approxi- 
mate belief state of size 1. The state wherein the VDU 
is on is placed into the belief state. The true state 
wherein the VDU is failed is discarded. At time 1, the 
VDU is commanded to open its valves. Since the only 
state in the belief state assumes the VDU is on, the sin- 
gle state in the updated belief state has the VDU on and 
all valves open. In the true, untracked state the valves 
are closed, as they never received a command. At time 
2, the pump is turned on. Pressure is observed at the 
outlet of the pump, while no pressure is observed down- 
stream of the valves. Failure of the pump alone has zero 
probability, given the observations. Failure of the VDU 
in the current time step has no effect on the valves. 
Thus, the most likely next state consistent with the 
observations requires that all valves spontaneously and 
independently shut. Regardless of the number of valves 
and the unlikeliness of spontaneously closure, this tran- 
sition must be taken if it exists. If it does not exist, the 
belief state approximation becomes empty. 

In general, as the true state evolves, the tracked sub- 
set of states may need to undergo arbitrarily unlikely 
transitions in order to remain consistent with the ob- 
servations. While only one trajectory is tracked in this 
example, for any fraction of the trajectories that are 
tracked, an example can be constructed wherein the ac- 
tual state of the system falls outside the tracked fraction 
and the error in the approximation may become arbi- 


trarily large. VVV propose an alternative to comitting 
to a subset of the current belief state or maintaining 
an approximation of the entire belief state. We propose 
to maintain the information necessary to begin incre- 
mentally generating the current belief state in best-first 
order at any point in time. Since we do not update the 
entire belief state, we do not have a sufficient statistic, 
so a history must be maintained. We introduce a vari- 
able to represent every state variable, command and 
observation at every point in time and an algorithm for 
incrementally generating the exact belief state at any 
point. Requiring a set of variables that grows linearly 
with the history of the system seems impractical except 
for short duration tasks. We apply two approximations 
motivated by our experience modeling physical systems 
for Livingstone. First we introduce variables only when 
close interaction requires. This approximation is con- 
servative in that it may admit impossible trajectories, 
which are likely to be eliminated with further observa- 
tions, but it does not eliminate any possible trajecto- 
ries. We then summarize possible trajectory segments 
that are consistent with extended periods of system evo- 
lution. We are then able to generate an approximate 
belief state using a constant number of variables. The 
variables represent an exact model of system evolution 
over the recent past, an approximate model over the 
intermediate past, and a gross summarization over the 
more distant past. This allows assignment of the most 
likely past transitions to be revisited as new observa- 
tions become available. The fewest variables, and thus 
the least flexibility, are allocated to segments of the sys- 
tem trajectory that have remained consistent with the 
system's observed evolution for the longest time. 

In the following sections of the paper, we give the 
complete history representation followed by a simple, 
exact, and intractable algorithm for enumerating the 
belief state. We introduce several optimizations and 
approximations in order to gain tractability while main- 
taining the ability to revise assesments of past sys- 
tem evolution. Finally we describe the results of run- 
ning the algorithm on test scenarios developed applying 
Livingstone and describe upcoming demonstrations of 
the software on NASA spacecraft. 

Transition Systems 

We wish to represent the possible histories of a system 
composed of non-deterministic, concurrent automata 
given the commands issued to the automata and their 
output. We create a structure that allows incremental, 
best-first enumeration of all possible trajectories by ex- 
tending the formalism of Livingstone . In order to com- 
pactly represent the trajectories, we add a set of transi- 
tion variables that represent the non-deterministic tran- 
sitions each automaton may make at each time step. 
Each assignment to a transition variable has a likeli- 
hood representing the prior probability of the corre- 
sponding non-deterministic transtion occurring. One 
trajectory of the system is thus an assignment to each 
transition variable, and given the appropriate indepe- 



draco assumptions, the set of trajectories can be in- 
crementally enumerated in order of likelihood. In or- 
der to capture the feasible behaviors of the automata, 
we introduce a set of formulae Me describing the in- 
put/output mapping of the automata in each state, and 
a set of formulae Mr for describing the feasible transi- 
tions of the automata. 

Definition 1 A transition system S is a tuple 
(n,T,D,C, Me> < Vf r)* where 

• II is a set of state variables representing the state 
of each automaton. Let n denote the number of au- 
tomata and m denote the number of discrete, syn- 
chronous time steps over which the state is to be 
tracked. IT then contains mxn variables. TI t will de- 
note the set of state variables representing the state 
of the system at time step t. Each state variable y 
ranges over a finite domain denoted J(y). The vari- 
able representing the assignment to y at time step t 
is denoted y t . 

• V is a finite set of dependent variables. 

• C is a finite set of command variables. 

• T is a set of transition variables. There is one tran- 
sition variable for each state variable at each time 
point, denoted r y ^. Each value in the domain of r y? t 
is is assigned a probability. 

• State s t is an assignment to U t uTtUV t uCt 

• Me is a propositional formula over lit and T> t that 
specifies the feasible subset of the state space. A state 
is feasible if it makes an assignment to II*U V t that is 
consistent with Me- 

• Mr is a propositional formula lit, V t , Cu Tt and 
Ilt+i that specifies the feasible sequences of states. 
Specifically, Mr is a conjunction of formulae of the 
form: 

4>t A (r M = r*)=>j/ e +! = y * 
where <p t is a propositional formula over II*, V t and 
C t , and r m 6 S(r yit ). 

Example 3 We introduce a transition system to model 
a VDU and two valves. The variables corresponding to 
the VDU consist of a state variable vdu representing the 
mode (on, off ’ or failed ), the transition variable r V( / U) 
a command variable crndin representing commands to 
the VDU or its associated valves (on, off \ open , c/ose, 
none), and a dependent variable cmdout representing 
the command the VDU passes on to its valves (open, 
close , or none). The set of feasible states of the VDU 
is specified by the following formula 

vdu — on => ( crndin — open =» cmdout = open ) 

A ( crndin — close => cmdout = close) 

A (( crndin ^ open A crndin / close) 

=> cmdout ~ none) 
vdu — off => cmdout — none 
vdu — failed => cmdout — none 

together with formulate like (vdu ^ on) V (vdu ^ 
off) V (vdu ^ failed ) , . . . that assert that variables have 
unique values. The time step subscript is omitted, indi- 
cating that all clauses refer to variables within the same 


time step. The valves r 1 and <;2 each have a state vari- 
able of domain (open, closed, or stuck dosed), a tran- 
sition variable r vl and a dependent variable flow V i of 
domain (zero, nonzero). The feasible states of the vl 
are specified by the formula below. The feasible states 
of u2 are specified similarly. 

open => flow v i = nonzero 
vl s closed => flow v i = zero 
vi = failed => flow v \ - zero 


Mr for T vUu and r v i is as follows. r v > is as r v i . 


r Vi iu,t = nominal => 

vdut — off A cmdint — on 
vdut — off A crndin ^ on 
vdut — on A cmdint — off 
vdut — on A C7ndin £ ^ off 
vdut = failed 

r v du,t = fail^vdut-t i = failed 


=> vdut+i — on 
=> vdut + i = off 
=> vdu t +i = off 
=> vdut + i = on 
=> vdut + [ = failed 


r v i,t ~ nominal 

t?l £ = closed A cmdout t — open => 
= closed A cmdout t ^ open => 
vlt = open A cmdout t = closed => 
vie = open A cmdout t # close => 
vie = stuckclosed => 

T v i,t = s<ic£=>vl,t+i = stuckclosed 


ule + i = open 
ult+i = closed 
vle+i = closed 
vlt+i = open 
ult+i = stuckclosed 


Infinitesimals 

In general we must consider all trajectories when de- 
termining the likelihoods, or even relative likelihoods, 
of a set of states, as many unlikely trajectories may 
contribute probability mass to the same outcome state. 
The transition system representation will allow’ us to 
enumerate the most likely trajectories of a system in 
order. We would like to find a natural restriction on 
the form of prior probabilities of transitions such that 
there is a correspondence between the most likely tra- 
jectories we are able to identify and the most likely 
states in which we are interested. Our experience ap- 
plying Livingstone was that an ad-hoc, order of magni- 
tude probability scale was a sufficient representation for 
two reasons. First, the internal behavior of a machine 
is usually far less stochastic than its interaction with its 
environment. There is an expected or nominal behavior 
that a component will exhibit for a given state and in- 
put, with failure modes one or more orders of magnitude 
less likely. Second, precise estimates for these priors are 
often either inaccessible or unknown. However, the rel- 
ative plausibility of each failure mode during operation 
can be elicited quite easily. In this work, we formalize 
and capitalize on these characteristics of the priors by 
making use of infinitesimals (Goldszmidt ic Pearl 1992) 
to model the relative likelihoods of failures. 

An infinitesimal probability is represented by an in- 
fintesimally small constant raised to an exponent re- 
ferred to as the rank. The rank can be considered the 
degree of unbelievability. Intuitively, one would not 
consider a rank 2 infinitesimal believable unless all rank 
0 and rank 1 possibilities had been eliminated. Com- 
position of infinitesimals has many desirable properties. 


U A and D an.* independent events, fch m 

Runk(AD) = Rank(A) + Runk(D) 

Runk(AV D) = min(Rank(A)y Runk(B)) 

Thus an outcome that can occur through multiple inde- 
pendent events has rank i if one event has rank i and the 
remaining events, even if arbitrarily many, have ranks of 
i or more. This property allows us to consider only the 
most likely trajectories leading to a state: if a sequence 
of events of rank i ends in state Sj, then an arbitrary 
number of higher rank (i.e. less likely) trajectories lead- 
ing to 3j will not change its rank. Similarly, if state Sj 
is reached by a trajectory of rank i, and no trajectory 
of rank i or less reaches s*, then 3j is more likely than 
St, even if an arbitrary number of unlikely trajectories 
leading to s* remain unconsidered. We frame our algo- 
rithms in terms most likely trajectories, knowing that 
there is a correct correspondence to most likely states 
given the infinitesimal interpretation of the priors. 

Trajectory Identification 

Definition 2 A trajectory for S is a sequence of states 
so* si , . . . s m such that for all t such that 0 < t < m s t 
is consistent with M z and for all t such that 0 < t < 
(m - 1) s t U$(+i is consistent with Mr- 

Consider the problem of determining the state of a 
physical process modeled by a transition system S at 
each point in a trajectory s 0 . . . s m . The subset of the 
dependent variables T> whose assignment corresponds to 
a measurement from the process will be referred to as 
the observations, Q. We are given an assignment for the 
initial state, IIo- In addition we are given assignments 
to commands C t and observations O t for all 0 < t < m. 
The task is to choose assignments to r y , f for all y and 
t so as to ensure consistency with Mz and Mr and 
maximize the likelihood of the trajectory. That is to 
say, given a starting state, a set of commands and a set 
of observations, we must find the most likely sequence 
of transitions such that each state is consistent with the 
state model M z and the transitions are consistent with 
the transition model Mr- We define the likelihood of 
the trajectory to be: 

m a 

^2 Rank ( T *‘ ) 

t = Q 

This definition makes the assumption that the likeli- 
hood of assignments to r Vi( are independent of r Xti . 

A Simple Tracking Algorithm 

The transition-system formulation suggests an intuitive 
procedure to begin enumerating the belief state at any 
point. The transition system is initialized with Mz 
and a copy of all variables, representing the initial state. 
At time step £, we introduce a copy of M z and a copy 
of all variables, representing the next state of the sys- 
tem, as well as a copy of Mr representing the con- 
straints between the current state and the next state. 
We assign C t and O t +i according to how the system 
was commanded and the observations that resulted. 


Example 4 Below is ;ui example trajectory-tracking 
problem. Tilt* command is cmdin and the observations 
are fluw V [ and fl(iw vi . These variables are assigned 
by the problem, as is the start state. The r yt assign- 
ments must be chosen. The remaining variables will 
be constrained based upon these assignments. For all 
r Uit we will assume Rank{r yA = nominal) = 0 and 
Rank[r yt ^ nominal ) == I. 


Variable 

t = 0 

t = 1 

t = 2 

vdut 

off 



cmdi i\t 

on 

open 

none 

cmdoutt 

vlt 

closed 



Tvl,l 

flow vlu 

zero 

zero 

zero 

vlt 

closed 


\ 

Tv2,t 

fl0W v 2,t 

zero 

zero 

zero 


Trajectories may be enumerated in order by enumer- 
ating assignments to all in order of the sum of 
the ranks, then testing for consistency with Mr and 
Mz • Conflict-directed, best-first search, or CBFS 
(Dressier & Struss 1992; de Kleer & Williams 1989; 
Williams & Nayak 1996) greatly focuses this process by 
using conflicts. A conflict that renders a candidate solu- 
tion inconsistent is used to avoid generating any further 
candidate solutions that contain the same conflict. 

Example 5 Below r are two solutions to the above prob- 
lem, representing a single failure of rank 1 at time 1 and 
a double failure of rank 2 at time 2, respectively. 


Variable 

t = 0 

t = 1 

t = 2 

vdut 

off 

failed 

failed 

Tvdu,t 

fail 

nominal 


cmdin t 

on 

open 

none 

cmdoutt 

none 

none 

none 

vlt 

closed 

closed 

closed 


nominal 

nominal 


flow v i t t 

zero 

zero 

zero 

vlt 

closed 

closed 

closed 

T V 2 ,i 

nominal 

nominal 


fl0Wvl,t 

zero 

zero 

zero 


Variable 

t = 0 

t = 1 

t = 2 

vdut 

Off 

on 

on 

Tvdu,t 

nominal 

nominal 


cmdint 

on 

open 

none 

cmdoutt 

none 

open 

none 

vlt 

closed 

closed 

stuckclosed 

T v l,t 

nominal 

stick 


f low V l,t 

zero 

zero 

zero 

vlt 

closed 

closed 

stuckclosed 

Tvl.t 

nominal 

stick 


fl0W v 2,t 

zero 

zero 

zero 


While applying CBFS to the full transition system 
exactly enumerates the most likely trajectories, and 
thus states, in order, problem size is a significant is- 
sue. Testing consistency of each candidate trajectory 
requires propositions representing the possible assign- 
ments to each variable at each time point. In addi- 
tion, these propositions are constrained by a copy of 
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Figure 3: Evolution before commanding the valves 

Mr and Mz at each time step. Let | n \ p denote 
the number of propositions needed to represent each 
possible value of each variable in II. If we wish to 
track the system for m steps, checking a single tra- 
jectory candidate becomes a consistency problem of 
m x (| T |p + | n |p + | C |p + | V | p ) propositions and 
77i x (| Mr | 4* | Mz I) clauses. Given sufficient time, 
m will outgrow the available computational resources. 

Problem Size Reduction 

In this section, we reduce the structure needed to rep- 
resent the evolution of the system at a time point from 
a complete copy of the system model to a small num- 
ber of variables and clauses. Intuitively, when a com- 
mand is issued to the system, only a small number of 
components participate in transmitting that command 
through the system or transitioning in response to the 
command. Consider Figure 3. The squares represent 
state variables, the lines sets of constraints from Mr- 
As of time 7, the valves, pump and VDU have not been 
commanded nor have they interacted with other compo- 
nents by passing a command. If we did not detect a fail- 
ure of any of these components, we represent the pos- 
sibility that they remained idle or failed silently with a 
single set of variables and constraints as illustrated. At 
time 7 we command the valves on. We require variables 
vis and v2$ to represent the new states of the valves. 
Mr suggests vdur vlj and v2j will interact with ulg 
and u2g. These variables, along with necessary transi- 
tion variables T 0 d U j, r vi j and, t v2 j, are introduced to 
the system with the appropriate clauses from Mr- For 
all other variables, the variable representing t/ 7 is ade- 
quate to represent yg. Figure 4 illustrates this process. 
In order to have a well-founded algorithm, we first we 
place a natural restriction on Mr that does not im- 
pact correctness. Second we introduce an approxima- 
tion that, importantly, does not rule out trajectories. 
Instead, some trajectories that are not consistent with 
past observations may be admitted, with the possibility 
that future observations will eliminate them. 

Restricting Mr 

We restrict Mr as do Livingstone and Burton 
(Williams ic Nayak 1997): a component moves to a 
failure state with equal probability from any state, and 
except for failures a component is kept in its current 
state by the idle command. Mr is limited to the forms: 

( T y,t = ^failure) ^ J/t+1 * Vfaiturt 


Figure 4: Evolution upon commanding the valves 

{C y j = idle) A (r yU - nominal) => y t + 1 = y t 
{C y ,t - c m ) A fa A (r y ,< = nominal) => y t +i - y* 

where fa is a propositional formula over Yl t uV t , C * € 
6(C v j)y nominal € J(r y ,*) and T/ a n € S(r ytt ). Formu- 
lae of the first form model failures while formulae of the 
third form model nominal, commanded transitions. We 
next preprocess fa by replacing references to T>t with 
an implicate n t that involves only II* . Intuitively, we re- 
place a formula on the command a component c receives 
with a formula on the chain of components causing c to 
receive the command. We expect that for the type of 
clauses Mr contains, growth will be proportional to the 
length of the component chain, /, which ranged from 1 
to 5 in the spacecraft model of (Bernard et ai 1998). 
Our initial experience supports this hypothesis. Non- 
idle, non-failure clauses then take the following form, 
which does not depend upon T>. 

{C y ,t = C*) A 7r f A (r y , t = nominal) => y t +i = y * 

Intuitively, as far as Mr is concerned, we need only 
introduce the variables of n t found in n t if C y>t ^ idle . 

Eliminating intermediate observations 

Mz remains, and requires introduction of all variables 
in n< and Vt in order to check consistency against Ot- 
We proceed by eliminating all variables O t for values of 
t sufficiently far in the past. That is to say, transition 
choices are only constrained by consistency between the 
trajectories they imply and recent observations. As 
the system evolves, variables representing older obser- 
vations, and the copies of Mz that constrain them, 
are discarded, or with relabeling, never introduced. For 
the portions of the trajectory where Mz is not intro- 
duced, we need only introduce the limited portion of 
lit required by Mr- This is of course an approxima- 
tion. Note that even after observations are discarded, 
no partial assignment to T that was discovered to be 
in conflict with the observations will be reconsidered, 
as such conflicts are stored. However, if a partial as- 
signment to T is in conflict with an observation, but is 
not considered until after the observation has been dis- 
carded, “imposter” trajectories containing the inconsis- 
tent assignment will be admitted. This is a conserva- 
tive approximation in that no consistent trajectories are 
eliminated. Each imposter trajectory is checked against 
new observations and eliminated as soon as it fails to 
describe the future evolution of the system. 




Selective Model Extension 

B<us(*(i upon these restrictions, the procedure extend in- 
troduces into time step t oidy the small fraction of the 
model involved with the evolution of the system due to 
the command C y<t = C m . This hinges upon Theorem 1. 

Theorem 1 Assume C yJ = C , C* £ idle, and for all 
x £ !J' = idle. Consider the formula of Mr 

(C y> t = C*) A 7 r t A (r y j = nominal) => y y+i = y* 

For a// state variables x tf x ^ y, if x t $ 7r t , then an 
equivalent consistency problem is formed by replacing 
Xt, and all formulae of Mr involving these variables 
with a constraint between Xt-i and Xt-i . 

Intuitively, Mr and the assignment to C t require xt+\ 
to be a failure or equal to x t -i and prevent x t from 
influence any other variables. Use of extend renders the 
problem size per time step proportional to [ n t |. Details 
appear in the eight-page version of this paper. 

Conflict Coverage Search 

The strengths of efficiently tracking a partial belief state 
are merged with the flexibility of incrementally enumer- 
ating belief states in the CoverTrack procedure of Fig- 
ure 5 . CoverTrack maintains a partial belief state of 
all consistent trajectories of rank 7. As a command 
and observations are received, trajectories are simply 
extended by the nominal, zero rank transition, and lit- 
tle computation is required. An extended trajectory 
that is inconsistent requires an additional failure, and 
will not have rank 7. These are discarded in keeping 
with the plausibility interpretation of infinitesimals. If 
no trajectories remain, a new belief state consisting of 
all trajectories of 7 + 1 is generated, and tracking re- 
sumes. CoverTrack uses the extend algorithm to ensure 
the transition system is extended by a small number of 
variables at each time step. As trajectories are elimi- 
nated, the conflicts between partial assignments to T 
and observations are recorded. The GenerateCover al- 
gorithm generates all assignments to T of rank 7 + 1 
(or higher if none exist) that cover all known conflicts. 
Intuitively, we leave the at their zero rank values, 
introducing reassignment only to avoid conflicts, with 
a toted cost of 7 + 1. This is the hitting set problem. 
Details appear in the eight-page version of this paper. 

Finite Horizons 

While selective extension reduces the variables per time 
step, we still require an unbounded number of variables 
over time. Note that members of the belief state of 
CoverTrack contain initial transition assignments that 
have remained consistent with the system's evolution 
for an extended period. We make an additional ap- 
proximation by comitting to these partial assignments. 
To operate over a fixed time horizon h, the most likely 
partial trajectories represented by assignments to T Vti 
for 0 < t < (m - h) are summarized by assignments 
to a single summary variable. The problem is reduced 


procedure Cover Track{) 

Conflicts - 0 ; 7 = 0; 

Variables = IIo u A> 

Assign U 0 to initial state; 

Belief State = the empty trajectory; 
loop 

while BehefState is not empty do 

Variables^ extend( Variables, C t ), adding T t 
Assign O t + i according to observations received; 
Assign Tt to nominal, 0 rank assignment. 
Survived = 0 ; 

while BehefState is not empty do 
Extension = pop (BehefState) + Tt] 
if consistent (Frtenjjon) then 
push( Extension, Survived); 

else 

push(confiict in Extension, Conflicts)-, 
endif 
endwhile 

BehefState = Survived; report BeliefState\ 

endwhile 

BehefState = Generate Cover (Variables, Conflicts ,7); 

7 = Rank ( first {BehefState)) + 1; 

endloop 

Figure 5 : Conflict Coverage Tracking Procedure 

to a constant size, wherein the last variable assignment 
captures a choice of likely initial trajectories. Details 
appear in the eight-page version of the paper. 

Results 

The algorithms presented have been implemented and 
correctly track scenarios translated from Ltvingstone 
that confound partial belief state algorithms. Exam- 
ples include silent failures whose impact propagates 
forward through time and multiple failure modes that 
are indistinguishable without future observations. We 
have largely translated Livingstone's spacecraft mod- 
els to begin performance analysis. Since Livingstone is 
roughly a Lisp implementation of CBFS with observa- 
tion and summary horizons of 1, we expect to meet or 
exceed its generally high performance at those settings. 
Interestingly, increasing correctness by adding variables 
to admit failures in the recent past will not uniformly 
degrade performance. Removing a particularly critical 
failure in a complex Livingstone model has increased 
computation by orders of magnitude when the failure 
occurred, as not admitting the failure forces considera- 
tion of incorrect and potentially expensive hypotheses. 
An analysis with large models over a variety of horizons 
will appear in a longer version of this paper. 

Related Work 

The problem described is a partially observable Markov 
decision process, or POMDP, with focus placed upon 
belief revision. A large body of work exists addressing 
belief revision of exact and approximate belief states. 
Boyen and Roller (Boyen & Roller 1998 ), for example, 



provide an approximate, factored belief state with a 
bounded error that can be updated without enumerat- 
ing the state space. Intuitively, the error bound relies 
upon the stochasticity of the underlying system, pa- 
rameterized by the problem’s mixing rate, to continu- 
ally smear both the approximate and true distributions, 
exponentially reducing rather than compounding errors 
over time. Unfortunately, the systems we consider have 
inadequate mixing rates. Intuitively, when monitoring 
the internal state of a complex device such as a space- 
craft, the device may behave as if it were deterministic 
for long periods, then exhibit a failure, then return to 
apparent determinism. There is no process in place 
with sufficient stochasticity to quickly contract an ar- 
bitrary error introduced by a factored approximation. 

Conclusions 

This paper presents incremental belief state genera- 
tion as an alternative to belief revision. Application of 
the described approximations creates a family of rep- 
resentations that track against a full model for a num- 
ber of steps, then against a reduced model, then sum- 
marize over the most likely initial trajectories. Since 
the abstractions of the trajectory segments (full, mini- 
mal extension or summary) are represented uniformly, 
a single, simple search procedure may be employed. 
Cover Track combines the efficiency of partial belief 
state propagation with the flexibilty of the transition 
system representation. The system will be evaluated 
on Earth-bound testbeds representing an interferome- 
ter and a Mars propellant plant. In addition, it will 
be flown as an experiment on the X-34 rocket plane in 
2001 and the X-37 orbital vehicle in 2002. 
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